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REMARKS 

The 21 March 2002 official action addressed claims 2-3 and 5-10. 
Claims 2-3 and 5-10 and amended and new claims 11-18 are added. Claims 2- 
3 and 5-18 are pending for reconsideration. 

Claim amendments 

All of the prior claims (claims 2-3 and 5-10) are amended to more 
specifically recite that cameras produce images of an object from different 
viewing angles, and that the images have different resolutions such that pixel 
units of the images are different in the amount of object represented thereby. 
All of the prior claims are further amended to indicate that at least one image is 
converted such that the pixel units of all images represent the same amount of 
the object. The prior claims are further amended to more clearly recite that a 
distance is calculated using triangulation (eliminating use of the equivalent term 
"stereo imaging" as previously recited). 

In addition, claims 5, 6, 9 and 10 are amended to more clearly recite that 
an image is converted by selecting from among a set of samples of the image 
that are sampled beginning at successive positions in the image (the second 
embodiment described in the application). 

In addition, new claims 11-18 are added. These are dependent claims 
that recite that the converting performed in claims 2, 3, 5 and 6 involves either 
sampling of images to reduce their resolution (as described in the first 
embodiment), or interpolating images to increase their resolution (as described in 
the third embodiment). 

No new matter is added. 
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Prior art rejection 

All claims were rejected under 35 USC § 103(a) as being obvious over 
Auty (U.S. 5,809,161) in view of Subbarao (U.S. 5,193,124). 

The claimed invention is not obvious in view of the teachings of Auty and 
Subbarao. The following presents applicant's positions regarding the cited 
references. 

Teaching of Auty 

Auty discloses a system for making detailed images of vehicle license 
plates. As shown in Figure 1, the system uses a first camera 6 for imaging 
vehicles at a distance, and a second camera 8 for making a detailed image of a 
vehicle when it is within a fixed range. The first camera 6 is used to determine 
when a vehicle is in the range of the second camera 8 so that the second 
camera 8 can take a high resolution image of the vehicle license plate. The first 
camera 6 provides periodic images that are used to track the approach of 
vehicles toward the second camera 8. Auty's system processes the images 
taken by the first camera 6 to determine when a vehicle is located in the range 
of the second camera 8 and then instructs the second camera 8 to make an 
image. 

Auty does not perform any conversion on images of the first and second 
cameras so that their pixel units are equal in the amount of object represented. 
It is true that Auty performs scaling of images made by the first long range 
camera 6 (col. 21, line 8). However, nothing in Auty indicates that this scaling 
is performed with respect to an image made by the second camera 8 so that the 
pixel units represent equal amounts of object, and there is no reason that such 
scaling would be performed, since Auty's second camera is not used at all in the 
tracking of vehicles performed by the first camera. Therefore there is no need 
for the images of the two cameras to be scaled to have equal pixel units, and 
that explains why Auty does not teach or suggest performing such scaling. 

Auty also does not disclose the concept of computing a distance to an 
object by triangulation. Auty's vehicle position tracking (col. 7 lines 1-39 and 
col. 20 line 29 - col. 21 line 63) uses only the image from the long distance 
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camera 6 and converts the position an object in the image into a position on the 
X - Y plane of the road surface (Fig. 4) based on the fixed relationship of the 
camera relative to the road surface. Auty does not use its two cameras to 
produce two different images of an object so that the distance to the object can 
be computed by triangulation as claimed. 

Therefore, while Auty's system includes two cameras that have different 
resolutions, this is the only similarity to the claimed invention. Auty's cameras 
are used in completely different manners for completely different purposes than 
the claimed invention. 

Teaching of Subbarao 

Subbarao discloses a system for determining the distance to an object 
based on focal lengths and aperture settings. Subbarao determines distance by 
focusing a camera on an object, an then reading the focal length of the camera 
to determine the distance to the object. Subbarao provides further accuracy by 
changing the aperture of the camera, which changes the depth of field (i.e. a 
range behind and in front of the exact focal distance from the lens, within which 
objects will appear to be in focus) and thus allows Subbarao to determine 
distance with greater accuracy. 

Subbarao's preferred embodiment uses a single camera with switchable 
lenses that enable the camera to have different focal lengths (Fig. 1, col. 14 
lines 33 - 41). In an alternative embodiment, Subbarao's system may use two 
cameras with two different focal lengths (Fig. 4), however those cameras 
produce images made from along the same line of sight, not from different 
viewing angles as claimed. As shown in Figure 4, the light emanating from the 
object "Scene" is received at a half-silvered mirror, which distributes it to both 
of cameras 1 and 2. It is clear that the image produced by each of the cameras 
is produced from exactly the same viewing angle. Imaging along a single line is 
in fact required for Subbarao's method, since the method measures the distance 
along that line to the object. 

Because Subbarao's system uses only one camera or two cameras that 
image along the same line of sight, Subbarao does not calculate a distance to an 



015.550122.1 



9 



Serial No. 09/473,015 



Attorney Docket No. 067183/0157 



object by triangulation. Triangulation requires the use of images made from two 
different visual angles, knowledge of those angles relative to some reference, 
and knowledge of the distance between the two locations from which the two 
different visual angles are viewed, so that computations can be made using 
those angles and distances to determine the distance to the object. In the case 
of Subbarao, the images are made along exactly the same line, and so there are 
no angles that can be used to compute distance using triangulation. 

It is also noted that Subbarao does not use two cameras having different 
resolutions such that the pixel units are different in the amount of object 
represented. Although it has been asserted in past rejections that changes in 
aperture are related to resolution, those actions simply assert that aperture 
affects resolution, and do not address the fact that Subbarao does not disclose 
two cameras having different resolutions such that the pixel units are different 
in the amount of object represented. 

Traversal of the rejection 

The present rejection essentially states that if the person of ordinary skill 
is given the following: 

1) a traffic monitoring system that uses a low resolution camera to track 
an incoming vehicle and a high resolution camera to take its picture when it 
reaches a certain point; and 

2) a distance measuring system that measures distance based on focal 
length and aperture, using two cameras that make images of the object along a 
single line of sight, 

then it would be obvious to the person of ordinary skill to implement a 
distance measuring system that makes images of the object from different visual 
angles using cameras with different resolutions, converts the images so that 
their pixel units represent the same amount of object, and then processes the 
images using triangulation to determine a distance to the object. 

The proposition is clearly unreasonable, yet this is the basis of the present 
rejection. 
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It is well known to patent applicants and patent examiners that the test 
for obviousness requires a determination of whether the differences between the 
claimed invention and the teachings of the prior art are such that it would be 
obvious for one to implement all of those differences based on what is taught in 
the prior art references. The following table shows the differences between 
claim 7 (used as an example) and the cited references: 



Claim 7 


Auty 


Subbarao 


A three-dimensional 
structure estimation 
apparatus which measures 
a distance to an object, 
comprising: 






a plurality of cameras 
for producing images of the 
object from different 
viewing angles, the 
cameras having different 
resolutions from each other 
such that pixel units of the 
images are different in the 
amount of object 
represented thereby; 




Subbarao uses two 
cameras for producing 
images of an object from 
the same visual angle. 
Subbarao's pixel units are 
not different in the amount 
of object represented 
thereby. The cameras are 
the same in all respects 
except for aperture and 
focal length. 


a conversion unit for 
converting at least one of 
the images outputted from 
said plurality of cameras 
such that the pixel units of 
all images are equal in the 
amount of object 
represented thereby; and 


Auty scales of the image of 
the first camera but does 
not say why. The scaling 
is not done with reference 
to the image made by the 
second camera, nothing in 
Auty indicates that Auty 
scales to make the pixel 
units of the first and 
second cameras the same, 
and there is no need for 
Auty to do so. 


Subbarao does not convert 
images so that the pixel 
units of all images are 
equal. 


a depth image 
production section for 
processing the images 
using triangulation to 
calculate a distance to the 
object. 


Auty does not calculate a 
distance to an object using 
triangulation. Auty 
determines distance using 
the image from a single 
camera. 


Subbarao does not 
calculate a distance using 
triangulation. 



From the table it is seen that the only similarity of either of the references 
to the present claims is that Auty uses two cameras having different 
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resolutions. As explained above, those cameras are used in a completely 
different way for a completely different purpose, and therefore Auty suggests 
nothing of the presently claimed invention. Subbarao does not teach any of the 
features of the claimed invention. Therefore there is no explicit suggestion in 
the cited references that would lead one of ordinary skill to implement all of the 
differences between the claimed invention and the cited references. Since the 
differences between the references and the claims are so significant, it would be 
unreasonable to assert that the suggestion comes from implied but unwritten 
aspects of the references. 

The foregoing remarks address the features of claim 7, and are equally 
applicable to claim 2 The remaining claims recite additional features in addition 
to those of claims 2 and 7 that are likewise not taught by the cited references. 
Claims 3 and 8 recite the additional features of using cameras having different 
fields and processing those images to have equal pixel units. Claims 5 and 9 
recite the additional feature of converting images by selecting from among a set 
of samples of the image that are taken beginning at successive points of the 
image. Claims 6 and 10 recite the additional features of both claims 3 and 8 
and claims 5 and 9. Claims 11, 13, 15 and 1 7 recite the feature that images 
are converted by sampling to reduce their resolution. Claims 12, 14, 16 and 18 
recite the feature that images are converted by interpolation to increase their 
resolution. None of these features is taught or implied by the cited references. 
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The foregoing amendments and remarks address all bases for objection 
and rejection and are believed to place the case in condition for allowance. The 
examiner is invited to contact the undersigned to resolve any remaining issues. 



Respectfully submitted, 



Date: August 1 6, 2002 
FOLEY & LARDNER 
Washington Harbour 
3000 K Street, N.W., Suite 500 
Washington, D.C. 20007-5109 
Telephone: (202) 672-5407 
Facsimile: (202) 672-5399 



By 



Ronald Coslick 
Attorney for Applicant 
Registration No. 36,489 
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Sir: 



In reply to the 21 March 2002 official action, the application is amended 
as follows: 

In the claims: 

2. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of cameras for producing images of the object from different 
viewing angles, afre kthe cameras having different resolutions from each other 
such that pixel units of the images are different in the amount of object 
represented thereby ; 

conversion means for converting at least one of the images outputted 
from e ach of said plurality of cameras i nto convert e d i mag e s whoso such that 
the pixel units of all images are equal in the amount of object represented 
thereby; and 
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a depth image production section for compar i ng processing the oonvort e d 
images using stereo i mag i ng triangulation to calculate a distance to the object. 

3. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of first cameras for producing images of the object from 
different viewing angles, af»d -the first cameras having different resolutions from 
each other such that pixel units of the images are different in the amount of 
object represented thereby ; 

a plurality of second cameras for producing images of the object from 
different viewing angles, aftd- the second cameras having different visual fields 
from each other such that pixel units of the images are different in the amount 
of object represented thereby ; 

conversion means for converting at least one of the images outputted 
from said first cameras and at least one of the images outputted from said 
second cameras i nto converted i mages whos e such that the pixel units of all 
images are equal in the amount of object represented thereby; and 

a depth image production section for compar i ng processing the converted 
images using st e r e o i maging triangulation to calculate a distance to the object. 

5. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of cameras for producing images of the object from different 
viewing angles, af>d -the cameras having different resolutions from each other 
such that pixel units of the images are different in the amount of object 
represented thereby ; 

conversion means for converting at least one of the images produced by 
each of said p l ura li ty of cameras i nto convorted i mag e s whos e such that the 
pixel units of all images are equal in the amount of object represented thereby, 
by para lle l movement by different movomont amounts the conversion means 
converting the at least one image by selecting from among a set of samples of 
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the at least one image that are sampled beginning at successive positions in the 
at least one image ; and 

a depth image production section for compar i ng processing the converted 
images using stereo i mag i ng triangulation to calculate a distance to the object. 

6. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of first cameras for producing images of the object from 
different viewing angles, aftd -the first cameras having different resolutions from 
each other such that pixel units of the images are different in the amount of 
object represented thereby ; 

a plurality of second cameras for producing images of the object from 
different viewing angles, af*d -the second cameras having different visual fields 
from each other such that pixel units of the images are different in the amount 
of object represented thereby ; 

conversion means for converting at least one of the images outputted 
from said first cameras and at least one of the images outputted from said 
second cameras into converted imag e s whos e such that the pixel units of all 
images are equal in the amount of object represented thereby^ by para ll el 
movomont by diff e r e nt mov e m e nt amounts the conversion means converting 
each of the at least one images by selecting from among sets of samples of 
each of the at least one images that are sampled beginning at successive 
positions in each of the at least one images ; and 

a depth image production section for comparing processing the converted 
images using st e r e o i mag i ng triangulation to calculate a distance to the object. 

7. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of cameras for producing images of the object from different 
viewing angles, aad -the cameras having different resolutions from each other 
such that pixel units of the images are different in the amount of object 
represented thereby ; 



015.550009.1 




Serial No. 09/473,015 Attorney Docket No. 067183/0157 

a conversion unit for converting at least one of the images outputted from 
eaoh of said plurality of cameras into converted i mages whose such that the 
pixel units of all images are equal in the amount of object represented thereby; 
and 

a depth image production section for compar i ng processing the conv e rt e d 
images using st e r e o i m a g i ng triangulation to calculate a distance to the object. 

8. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of first cameras for producing images of the object from 
different viewing angles, aed -the first cameras having different resolutions from 
each other such that pixel units of the images are different in the amount of 
object represented thereby ; 

a plurality of second cameras for producing images of the object from 
different viewing angles, anc^- the second cameras having different visual fields 
from each other such that pixel units of the images are different in the amount 
of object represented thereby ; 

a conversion unit for converting at least one of the images outputted from 
said first cameras and at least one of the images outputted from said second 
cameras i nto conv e rt e d i mages whos e such that the pixel units of all images are 
equal in the amount ef-of object represented thereby; and 

a depth image production section for compar i ng processing the convert e d 
images using ster e o i mag i ng triangulation to calculate a distance to the object. 

9. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of cameras for producing images of the object from different 
viewing angles, aftd- the cameras having different resolutions from each other 
such that pixel units of the images are different in the amount of object 
represented thereby ; 

a conversion unit for converting at least one of the images produced by 
eaoh of said plura li ty of cameras i nto converted i mag e s whos e such that the 
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pixel units of all images are equal in the amount of object represented thereby, 
by para llel movement by d i ff e r e nt mov e mont amounts the conversion means 
converting the at least one image by selecting from among a set of samples of 
the at least one image that are sampled beginning at successive positions in the 
at least one image ; and 

a depth image production section for compar i ng processing the converted 
images using otoroo i maging triangulation to calculate a distance to the object. 

10. (Amended Four Times) A three-dimensional structure estimation 
apparatus which measures a distance to an object, comprising: 

a plurality of first cameras for producing images of the object from 
different viewing angles, aed -the first cameras having different resolutions from 
each other such that pixel units of the images are different in the amount of 
object represented thereby ; 

a plurality of second cameras for producing images of the object from 
different viewing angles, afid -the second cameras having different visual fields 
from each other such that pixel units of the images are different in the amount 
of object represented thereby ; 

a conversion unit for converting at least one of the images outputted from 
said first cameras and at least one of the images outputted from said second 
cameras i nto converted images whose such that the pixel units of all images are 
equal in the amount of object represented thereby ^ by para l le l mov e m e nt by 
d i ff e r e nt movement amounts the conversion means converting each of the at 
least one images by selecting from among sets of samples of each of the at 
least one images that are sampled beginning at successive positions in each of 
the at least one images ; and 

a depth image production section for comparing processing the converted 
images using st e reo imag i ng triangulation to calculate a distance to the object. 

1 1 . (New) The apparatus claimed in claim 2, wherein the conversion 
means samples images such that the pixel units of sampled images represent an 
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amount of object represented by pixel units of an image having a lowest 
resolution. 

12. (New) The apparatus claimed in claim 2, wherein the conversion 
means interpolates images such that the pixel units of interpolated images 
represent an amount of object represented by pixel units of an image having a 
highest resolution. 

13. (New) The apparatus claimed in claim 3, wherein the conversion 
means samples images such that the pixel units of sampled images represent an 
amount of object represented by pixel units of an image having a lowest 
resolution. 

14. (New) The apparatus claimed in claim 3, wherein the conversion 
means interpolates images such that the pixel units of interpolated images 
represent an amount of object represented by pixel units of an image having a 
highest resolution. 

15. (New) The apparatus claimed in claim 7, wherein the conversion unit 
samples images such that the pixel units of sampled images represent an 
amount of object represented by pixel units of an image having a lowest 
resolution. 

16. (New) The apparatus claimed in claim 7, wherein the conversion unit 
interpolates images such that the pixel units of interpolated images represent an 
amount of object represented by pixel units of an image having a highest 
resolution. 

17. (New) The apparatus claimed in claim 8, wherein the conversion unit 
samples images such that the pixel units of sampled images represent an 
amount of object represented by pixel units of an image having a lowest 
resolution. 
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